A Central Limit Theorem for Temporally Nonhomogenous Markov Chains with Applications to Dynamic Programming

A Central Limit Theorem for Temporally Nonhomogenous Markov Chains with Applications to Dynamic Programming

We prove a central limit theorem for a class of additive processes that arise naturally in the theory of finite horizon Markov decision problems. The main theorem generalizes a classic result of Dobrushin for temporally nonhomogeneous Markov chains, and the principal innovation is that here the summands are permitted to depend on both the current state and a bounded number of future states of t...

A central limit theorem for adaptive and interacting Markov chains

G. FORT1,*,∗ , E. MOULINES1,**,∗ , P. PRIOURET2,† , P. VANDEKERKHOVE3,‡ LTCI, TELECOM ParisTech & CNRS, 46 rue Barrault, 75634 Paris Cedex 13, France E-mail: [email protected]; [email protected] LPMA, Univ. Pierre et Marie Curie, Bôıte courrier 188 75252 PARIS Cedex 05, France E-mail: †[email protected] LAMA, Univ. Paris-Est Marne-la-Vallée, Cité Descarte...

Variance Estimation in the Central Limit Theorem for Markov Chains

This article concerns the variance estimation in the central limit theorem for finite recurrent Markov chains. The associated variance is calculated in terms of the transition matrix of the Markov chain. We prove the equivalence of different matrix forms representing this variance. The maximum likelihood estimator for this variance is constructed and it is proved that it is strongly consistent ...

A central limit theorem for functions of a Markov chain with applications to shifts

Central Limit Theorems for Conditional Markov Chains

This paper studies Central Limit Theorems for real-valued functionals of Conditional Markov Chains. Using a classical result by Dobrushin (1956) for non-stationary Markov chains, a conditional Central Limit Theorem for fixed sequences of observations is established. The asymptotic variance can be estimated by resampling the latent states conditional on the observations. If the conditional means...